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Abstract 


Since  the  Defense  Reorganization  Act  of  1986,  the  military  has  endeavored  to 
beeome  more  integrated  and  better  able  to  perform  joint  operations.  The  United  States  Joint 
Forees  Command  (JFCOM)  was  established  to  lead  DoD’s  transformation  efforts. 
Experimentation  and  assessment  are  necessary  steps  in  the  development  and  aequisition  of 
new  hardware,  software,  proeedures,  and  doetrine.  Field  experiments  provide  the 
opportunity  to  perform  these  funetions  in  an  operational  environment.  Senior  DoD 
leadership  direeted  that  joint  training,  experimentation,  and  assessment  aetivities  be 
performed  simultaneously  in  order  to  take  advantage  of  potential  effieieneies.  The  planning, 
exeeution,  and  analysis  of  several  eombined  venues  to  date  has  revealed  that  many 
compromises  were  neeessary,  many  eonfliets  oeeurred,  and  stakeholders’  objeetives  were  not 
met.  Several  studies  have  been  undertaken  doeumenting  the  different  requirements  and 
methodologies  of  training,  experimentation,  and  assessment.  The  Direetor,  Operational  Test 
and  Evaluation,  has  produced  a  strategie  roadmap  to  improve  the  aequisition  eommunity’s 
integration  into  eombined  venues.  While  this  doeument  provides  reeommendations  that  will 
benefit  all  stakeholders,  it  is  biased  toward  improving  experimentation  and  assessments.  It  is 
ineumbent  upon  military  leadership  to  produee  an  equivalent  strategie  plan  that  addresses 
eommanders’  training  priorities.  This  paper  highlights  several  shorteomings  of  eurrent 
doetrine  and  execution  of  combined  venues  and  provides  reeommendations  to  improve  the 
effectiveness  of  joint  training. 
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INTRODUCTION 


Current  policy  and  practice  combine  experimentation  and  assessment  activities  with 
service/joint  training  events  such  as  Roving  Sands,  Red  Flag  and  Joint  Task  Force  Exercises. 
Many  arguments  have  been  made  of  the  merits  of  combining  training,  experimentation,  and 
assessment  activities.  However,  efforts  to  date  have  failed  to  prove  that  the  potential 
efficiencies  are  worth  the  required  compromises.  Current  doctrine,  guidance,  and  culture 
are  inadequate  to  fully  realize  the  potential  efficiencies  expected  by  combining  joint 
training,  experimentation,  and  assessment. 

The  inadequacies  of  combined  venues  have  resulted  in  shortcomings  in  many  areas  of 
execution  and  objectives  accomplishment.  If  not  resolved,  these  shortcomings  will  have 
potentially  profound  effects  on  the  Combatant  Commander  in  terms  of  force  readiness  issues, 
training,  and  assessments  of  equipment  and  procedures.  Without  the  ability  to  test, 
experiment,  and  collect  valid  data  in  realistic  environments.  Department  of  Defense  (DoD) 
experimentation  and  acquisition  programs  will  have  to  find  other  venues  to  perform  accurate 
assessments.  Based  on  current  doctrine  and  practices,  and  the  results  of  efforts  to  date,  the 
decision  to  combine  joint  training,  experimentation,  and  assessments  requires  further  study 
and  guidance. 

Current  doctrine  directs  the  incorporation  of  training,  experimentation,  testing, 
assessment,  tactics,  techniques,  and  procedures  (TTP)  development,  and  feedback  into  joint 
training  events.  Within  each  of  these  major  areas  are  sub-areas  —  service  training  vs.  joint 
training;  training  vs.  experimentation;  doctrinal  adherence  vs.  doctrinal  development; 
training  feedback  vs.  analytical  feedback;  and  force  readiness  assessment  vs.  venue 
assessment  vs.  measures  of  effectiveness/performance  assessment.  While  execution  of  these 
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sub-areas  employs  similar  environments,  methodologies  many  times  are  eontradietory 
ereating  eonfliets  and  requiring  eompromise.  For  the  purpose  of  this  paper,  these  efforts  will 
be  considered  under  three  general  headings;  training  (to  include  both  service  and  joint), 
experimentation  (to  include  both  equipment  and  doctrine,  and  testing  efforts),  and  assessment 
(to  include  participant  readiness,  joint  task  viability,  measures,  and  feedback).  Other  efforts 
such  a  modeling  and  simulation  (M&S)  and  the  Joint  National  Training  Capability  (JNTC) 
are  being  incorporated  into  training  events  to  support  all  these  efforts.  The  political  visibility 
of  M&S  and  JNTC  development  add  additional  and  significant  impacts  to  exercise  planning 
and  execution. 

The  concept  of  transformation  has  an  overarching  influence  on  all  aspects  of  training, 
experimentation,  and  assessment,  as  well  as  culture.  Like  any  large,  bureaucratic 
organization,  the  military  can  not  change  its  culture  as  quickly  as  it  can  change  doctrine.  An 
examination  of  current  doctrine,  guidance,  and  directed  studies  of  experimentation  reveal 
existing  shortcomings  that  will  require  further  attention  to  fully  realize  the  efficiencies  of 
combined  venues  while  maintaining  effective  training. 

THE  EVOLUTION  OF  JOINT  TRAINING,  EXPERIMENTATION,  AND 

TRANSFORMATION 

One  of  the  first  efforts  at  joint  cooperation  by  the  military  occurred  during  the 
planning  for  World  War  II.  U.S.  civilian  and  military  leadership  realized  the  potential  of 
joint  military  efforts  while  negotiating  and  planning  with  British  joint  military  leadership. 
However,  it  was  not  until  the  Defense  Reorganization  Act  (DRA)  of  1986  (Goldwater- 
Nichols  legislation)  that  the  military  began  any  serious  effort  to  integrate.  The  DRA  shifted 
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several  military  authorities  and  responsibilities  from  the  Service  Chiefs  to  the  Regional 
Combatant  Commanders.  In  addition,  the  DRA  mandated  that  the  military  develop  the 
capability  to  work  jointly.  The  U.S.  Atlantic  Command  (USACOM)  was  established  in  1993 
as  the  “Joint  Force  Integrator”  responsible  for  joint  training.' 

Under  the  new  Unified  Command  Plan  (UCP),  Atlantic  Command  assumed 
combatant  command  of  the  service  component  commands;  the  Army's  Forces  Command 
(FORSCOM),  the  Air  Force's  Air  Combat  Command  (ACC),  the  Marine  Corps’  Forces 
Command  Atlantic  (MARFORLANT),  and  the  Navy's  U.S.  Atlantic  Fleet  (later  designated 
Fleet  Forces  Command  (FLTFORCOM)).  In  October  1999,  the  name  of  Atlantic  Command 
changed  to  United  States  Joint  Forces  Command  to  emphasize  the  command's  role  leading 
transformation  of  U.S.  military  forces.  Joint  Forces  Command  (JFCOM)  gained  a  functional 
mandate  to  lead  transformation  of  U.S.  military  joint  warfighting  into  the  21st  Century. 

Since  1986,  the  services  made  some  effort  to  work  more  jointly.  However,  due  to  the 
structural  inertia  of  evolving  joint  systems  and  cultural  change,  early  joint  operations 
reflected  planning  and  operations  aimed  at  deconfiiction  vice  integration.  The  Office  of  the 
Secretary  of  Defense’s  (OSD)  introduction  of  the  concept  of  transformation  and  JFCOM’s 
efforts  to  fulfill  its  new  mandate  reinvigorated  the  services’  efforts  to  operate  in  a  joint 
environment. 

Experimentation  and  assessment  are  necessary  steps  in  the  development  and 
acquisition  of  new  hardware,  software,  procedures,  and  doctrine.  Field  experiments  provide 
the  opportunity  to  perform  these  functions  in  an  operational  environment.  Establishing  the 
appropriate  operational  environment  is  time-consuming,  manpower  intensive,  and  expensive. 
The  environment  required  for  experimentation  and  assessment  has  many  similarities  to  that 
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required  for  joint  training.  To  aehieve  effieieneies  in  time,  manpower,  and  funding,  an 
obvious  ehoiee  is  to  perform  training,  experimentation,  and  assessment  during  the  same 
event.  Senior  defense  leadership  agrees  with  this  philosophy  and  is  a  strong  proponent  for 
this  option.  In  the  2001  Quadrennial  Defense  Review,  DoD  leadership  identified  the  four 
pillars  of  transformation.  Three  of  the  four  direetly  impaeted  joint  training  exereises: 

-  Strengthening  joint  operations  through  standing  joint  task  foree  headquarters, 
improved  joint  eommand  and  control,  joint  training,  and  an  expanded  joint  forees 
presenee  poliey. 

-  Experimenting  with  new  approaehes  to  warfare,  operational  eoneepts  and 
eapabilities,  and  organizational  eonstruets  sueh  as  standing  joint  forees  through 
wargaming,  simulations  and  field  exercises  foeused  on  emerging  ehallenges  and 
opportunities. 

-  Developing  transformational  capabilities  through  inereased  and  wide-ranging 
seienee  and  teehnology,  seleetive  inereases  in  proeurement,  and  innovations  in  DoD 
processes  (emphasis  added). 

As  the  transformation  leader,  JFCOM  began  to  aggressively  implement  DoD  joint 
and  transformation  guidanee.  In  response  to  the  2001  National  Defense  Authorization  Aet, 
JFCOM  rushed  to  eonduet  Millennium  Challenge  (MC)  ’02  in  mid-2002.  MC  ’02  was  the 
largest  and  most  eomplieated  event  hosted  by  JFCOM  up  to  that  time.  This  venue  brought 
together  many  different  experiments  in  a  joint  environment.  Although  this  venture  was  met 
with  some  eritieism,  JFCOM  believed  that  it  was  on  the  right  traek  in  developing  venues  that 
eould  aeeomplish  all  of  DoD’s  transformation  objeetives. 


CURRENT  PRACTICES 

JFCOM  has  leveraged  several  different  training  and  experimentation  venues 
inereasing  their  seope  in  an  attempt  to  develop  the  appropriate  mix  of  training, 
experimentation,  and  assessment.  Tremendous  levels  of  effort  and  funding  have  gone  into 
the  planning,  exeeution  and  analysis  of  sueh  ventures.  To  prevent  adding  more  exereises  to 
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the  already  strained  operational  tempo  (OPTEMPO)  of  military  units,  JFCOM  developed  the 
coneept  of  integrating  into  the  serviee  exereises  of  its  component  commands;  FORSCOM’s 
Roving  Sands,  FlSFFTFORCOM’s  Joint  Task  Force  Exercise  (JTFEX),  and  ACC’s  Red 
Flag.  Previous  to  this  venture,  each  of  the  services  had  been  increasing  the  jointness  of  these 
exercises,  but  their  primary  focus  was  still  on  fulfilling  each  commander’s  specific  training 
requirements.  From  2003  to  2005,  JFCOM  sponsored  expanded  versions  of  these  training 
exercises  executed  by  their  respective  combatant  commands. 


CONFLICTS 

Under  Title  10  and  joint  doctrine,  the  services,  combatant  commanders,  JFCOM,  and 
the  Joint  Staff  have  shared  responsibilities  in  the  realm  of  training.  In  addition  to  equipping 
and  training  their  forces,  the  combatant  commanders  are  responsible  for  assessing  the 
readiness  levels  of  their  forces."^  JFCOM  is  the  lead  agent  responsible  to  the  Chairman  Joint 
Chiefs  of  Staff  (CJCS)  for  joint  force  training.  Shared  responsibilities  and  self-assessment 
requirements  present  potential  conflicts  during  the  planning,  execution,  and  feedback  of  joint 
training  venues.  Additionally,  some  training  venues  are  considered  to  be  one  of  the  final 
preparatory  phases  for  deploying  forces.  Real-world  contingencies  will  sometimes  require 
participating  forces  to  leave  an  exercise  early  or  to  not  even  be  available.  When  this  occurs, 
senior  leadership  has  to  reprioritize  objectives  to  support  real-world  commitments.  Current 
doctrine  and  the  incorporation  of  JFCOM  objectives  into  service-specific  training  venues 
create  conflicts  in  leadership,  priorities,  execution,  and  assessment. 

Although  training,  experimentation,  and  assessment  events  have  many  similar 
components  and  requirements,  satisfaction  of  the  objectives  of  each  requires  different 


5 


execution  methodologies.  However,  in  its  Testing  in  a  Joint  Environment  Roadmap: 
Strategic  Planning  Guidance  Fiscal  Years  2006-2011—  Final  Report,  the  DoD  stated: 

The  event  planning  and  execution  process  for  systems  engineering,  testing, 
training,  and  experimentation  events  have  much  in  common,  and  the  venues  where 
testing,  training,  and  experimentation  events  in  the  joint  context  are  conducted  are 
virtually  identical. . . .  Creating  separate  joint  venues  and  joint  mission  infrastructures 
for  systems  engineering,  T&E,  training,  and  experimentation  is  not  effective, 
efficient,  or  affordable.^ 

This  guidance  from  DoD  and  JFCOM’s  overwhelming  emphasis  on  transformation  are 
having  both  beneficial  and  detrimental  effects  on  joint  training  capabilities. 

In  most  training  scenarios,  conditions  are  established  to  ensure  interactions  take  place 
so  participants  can  practice  certain  procedures  and  techniques.  Depending  upon  the  exercise 
construct,  resources,  and  size  of  the  training  audience,  forces  may  only  practice  their 
procedures  until  they  demonstrate  the  desired  proficiency.  Usually,  the  forces  have  many 
tasks  they  must  practice  and  demonstrate  and  therefore  do  not  have  the  opportunity  to 
perform  numerous  repetitions  of  the  same  task.  During  execution,  the  pace  of  the  exercise 
(including  the  number  of  repetitions)  is  based  on  the  participants  meeting  minimum 
standards.  While  this  exercise  construct  satisfies  the  exercise  Joint  Force  Commander  (JFC) 
or  combatant  commander’s  objectives,  it  may  not  (and  in  most  cases  does  not)  satisfy 
experimental  or  assessment  criteria. 

Experimental  and  assessment  data  must  be  credible.  Depending  upon  the  degree  of 
statistical  significance  needed  for  analysis,  credibility  requires  making  enough  observations 
to  provide  meaningful  results.  As  mentioned,  exercises  rarely  have  enough  time  to  perform 
the  amount  of  repetitions  desired  by  experimentations  and  assessments.  Additionally, 
execution  management  decisions  are  made  to  account  for  weather,  real-world  commitments, 
range  limitations,  safety,  etc.  These  decisions  are  made  by  the  exercise  JFC  or  component 
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commander  and  are  usually  based  on  meeting  his  objectives;  henee,  when  necessary,  training 
requirements  will  take  preeedenee  over  experiments  or  assessments.  Task  repetition 
requirements  between  training,  experiments,  and  assessments  are  usually  dijferent  and  are 
ajfected  by  priority  objective  achievement  of  the  other  venues. 

Appropriate  feedback  mechanisms  must  be  used  to  aehieve  the  greatest  benefit  from 
training,  experimentation,  or  assessment.  Exercise  operations  must  be  observed  and  recorded 
in  order  to  provide  the  data  neeessary  for  feedback.  However,  training,  experimentation,  and 
assessment  all  have  different  requirements  and  purposes  for  feedbaek.  Consequently, 
feedback  data  vary  according  to  timeliness,  aceuraey,  preeision,  and  completeness.  Training 
feedbaek  should  be  timely,  accurate,  and  promote  improved  performance.  Experimentation 
and  assessment  feedbaek  is  usually  more  time-eonsuming  to  ensure  aecurate  analysis  and  is 
not  neeessarily  eoneerned  with  improving  partieipant  performance.  Many  training  exereises, 
especially  those  that  operate  on  24-hour  eycles,  are  not  able  to  provide  sufficient  time  or  the 
appropriate  participants  for  lengthy  feedbaek  sessions.  Current  mixed  venues  do  not  allow 
for  sufficient  feedback  to  support  training,  experimentation  and  assessment. 

The  instrumentation  and  infrastrueture  neeessary  for  eolleeting  feedback  data  is 
expensive  and  requires  extensive  planning.  It  also  requires  use  of  full  system  eapabilities  and 
comprehensive  threat  representation.  In  most  oases,  the  level  of  instrumentation  required  for 
experimentation  and  assessment  exceeds  that  required  for  training.  The  DoD  Command  and 
Control  Researoh  Program  desoribes  some  of  the  basic  principles  of  experimentation:  “Ideal 
experimentation  involves  valid,  reliable,  precise,  and  credible  measurement  of  all 
variables.”^  Interestingly,  training  feedbaek  benefits  signifioantly  when  it  uses  the  inoreased 
instrumentation  precision  and  fidelity  required  for  experimentation  and  assessment. 
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Participants  in  past  experimentation  and  assessment  exereises  frequently  eommented  that 
those  venues  provided  some  of  the  best  joint  training  they  had  ever  reeeived/  The  inereased 
preeision  of  instrumentation  was  just  one  aspeet  that  eontributed  to  the  inereased  level  of 
training  reeeived.  Support  for  instrumentation  and  data  collection  varies  among  the  different 
venue  types. 

Another  signifieant  eonfliet  in  venue  priorities  is  the  amount  of  leeway  partieipants 
have  in  following  established  doetrine  and  TTP.  Usually,  the  purpose  of  training  is  to  learn 
eurrent  doetrine  and  TTP.  Therefore,  fairly  striet  adherenee  is  neeessary  and  is  a  eriterion  for 
sueeess.  Experimentation  many  times  involves  using  new  or  developmental  doetrine  and 
TTP.  The  JFCOM  glossary  defines  Joint  Experimentation  (JE)  as  the  “applieation  of 
seientifie  experimentation  proeedures  to  assess  the  effeetiveness  of  proposed  (hypothesized) 

a 

joint  warfighting  eoneept  elements”  (emphasis  added).  CJCS  defines  experiments  as  “an 
iterative  proeess  of  collecting  developing  and  exploring  concepts...''’  (emphasis  added). ^ 
CJCS  Instruetion  3180.1  states  that  JECOM  will  eonduet  JE  to  “explore,  demonstrate,  and 
evaluate  joint  warfighting  eoneepts  (to  include  breakthrough  joint  concept  development)  ” 
(emphasis  added). Assessment  efforts  ean  involve  a  eombination  of  both  adherenee  and 
variation  to  doetrine  and  TTP.  The  mixture  of  the  use  of  current  and/or  developmental 
doctrine  and  TTP  can  affect  objective  accomplishment  among  the  various  venue  types,  as 
well  as  add  confusion  among  the  participants  and  observers. 

Serviee  training  ean  be  eonsidered  “undergraduate-level”  training  while  joint  training 
ean  be  eonsidered  “graduate-levef  ’  training.  Due  to  inereasing  OPTEMPO,  the  serviees  must 
sometimes  use  joint  training  events  as  an  opportunity  to  train  to  basie  undergraduate-level 
serviee  eapabilities.  Using  forees  that  are  minimally  profieient  in  serviee-speeifie  skills  has  a 
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degrading  affect  on  the  graduate-level  joint  training  environment  as  well  as  affecting 

experimentation  and  assessment.  As  stated  in  The  Role  of  Experimentation  in  Building 

Future  Naval  Forces,  .  .proficient  Service  core  capabilities  are  a  prerequisite  for  joint 

warfighting.”''  Additionally,  “...  perhaps  more  DOD  experiments  have  failed  to  achieve 

12 

their  goals  because  of  inadequately  trained  subjects  than  for  any  other  single  cause.” 

RESULTS  OF  RECENT  COMBINED  VENUES 

Although  official  statements  reflect  very  positive  results  of  recent  mixed  venue 
events,  post-exercise  documentation  has  painted  a  different  picture.  Each  of  the  conflicts 
mentioned  in  the  previous  section  have  been  present  in  one  or  more  of  the  following  recent 
combined  events. 

MC  ’02  was  criticized  as  being  too  scripted  and  did  not  adequately  assess  the  JFCOM 
concepts  of  rapid  decisive  operations,  effects-based  operations,  or  operational  net 
assessment.  After  many  months  of  planning  and  preparation.  Roving  Sands  03  was 
severely  downgraded  from  a  live-virtual-constructive  (LVC)  event  to  a  simulated  only  event 
due  to  troop  deployments.  Combined  JTFEX  (CJTFEX)  04-2  was  cited  as  not  being  capable 
of  producing  sufficient  data  to  meet  some  of  its  exercise  objectives  due  in  part  to  exercise 
construct  and  limitations,  and  the  participants’  ability  to  fulfill  event  objectives.  Part  of  the 
reason  for  the  lack  of  data  collection  was,  again,  the  response  to  real  world  commitments 
resulting  in  an  early  departure  of  the  participating  carrier  strike  group  (CSG).  Joint  Red  Flag 
05  also  had  several  issues  that  affected  training,  experimentation,  and  assessments. 
Conflicting  exercise  priorities  among  the  various  participants  and  scenario  inconsistencies 
between  FVC  operations  created  considerable  confusion  among  tactical  operators  and 
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delayed  the  aeeomplishment  of  several  learning  objeetives.  Immature  JNTC  software  and 
network  eonneetivity  hampered  the  ability  to  aequire  suffieient  assessment  data  and  degraded 
feedback  operations.'^ 

Some  of  the  issues  that  contributed  to  the  inability  to  meet  objectives  in  these  events 
were  the  lack  of  coordination,  common  objectives,  and  guidance  at  the  senior  levels. 

JFCOM  experimental  and  assessment  objectives  did  not  mesh  with  commanders’  training 
objectives.  Training  representatives  were  reluctant  to  allow  experiment  and  assessment 
representatives  to  become  involved  in  exercise  planning  specifics.  Experimentation  and 
assessment  organizations  have  programmed  significant  funding  and  manpower  in  order  to 
satisfy  their  objectives.  Many  experiments  have  timelines  that  must  be  met  and  many 
assessment  activities  support  ongoing  trend  analysis.  The  inability  to  meet  objectives  due  to 
insufficient  data  is  not  only  frustrating,  but  inefficient  and  can  have  serious  repercussions  to  a 
particular  program. 

These  training  venues  are  run  by  a  group  of  many  individuals  dedicated  to  making 
them  successful.  Unfortunately,  cultural  pressures  to  not  only  succeed,  but  to  excel  cause 
individuals  to  “look  out  for  number  one”  and  attempt  to  optimize  mixed  venues  to  support 
their  organization’s  own  objectives;  be  it  training,  experimentation,  or  assessment.  During 
exercise  planning,  a  significant  amount  of  coordination  and  compromise  is  required  because 
component  commander  training  exercises  are  not  designed  to  support  all  of  the  efforts  they 
are  being  asked  to.  The  result  is  that  recent  combined  venues  attempts  have  not  been  able  to 
satisfy  everyone’s  requirements. 

The  book.  Code  of  Best  Practice  for  Experimentation,  was  inspired  and  supported, 
incidentally,  by  MG  Dean  Cash,  the  then  JFCOM  J9,  responsible  for  joint  experimentation.'^ 
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This  book  describes  many  of  the  issues  that  arise  when  combining  training  and 
experimentation.  Some  of  the  more  notable  and  common  issues  are  participant  training 
levels,  hesitancy  of  participants  to  be  creative  when  being  evaluated,  insufficient  feedback 
time,  an  over-reliance  on  rapidly  produced  quick-look  reports,  DoD’s  unwillingness  to  admit 
and  report  failure  (thereby  declaring  most  events  successful  when,  in  fact,  they  were  not), 
and  the  long  analysis  timeframe  required  to  adequately  assess  data.  Additionally, 
commanders  “are  very  busy  and  under  great  pressure  to  move  ahead  rapidly.  Unfortunately, 
this  has  translated,  in  some  cases,  into  unrealistic  schedules  where  events  start  occurring  so 
rapidly  that  they  can  neither  be  managed  properly  nor  used  to  build  coherent  knowledge”^^ 
Finally,  it  discusses  DoD’s  culture  of  accepting  failure  and  its  impact  on  experimentation: 

Perhaps  the  strongest  indication  that  DoD  experimentation  remains  immature  is  the 
inability  to  recognize  the  value  of  permitting  “failure.”  Failure  is  either  (A)  a  failure 
to  support  the  hypothesis  or  novel  ideas,  or  (B)  a  failure  to  run  a  good  experiment. 
Currently,  many  in  DoD  will  not  accept  a  failure  of  Type  A,  but  often  accept  a  failure 
of  Type  B.  This  is  exactly  wrong  and  a  sign  that  these  individuals  do  not  understand 
experimentation.  This  attitude  arises  from  the  mindset  necessary  for  military 
success.^* 

The  General  Accounting  Office’s  (GAO)  study  titled  Military  Training  Actions 
Needed  to  Enhance  DoD ’s  Program  to  Transform  Joint  Training  (GAO-05-548),  found  that 
some  of  the  stakeholders  did  not  have  a  complete  understanding  of  training  transformation 
and  do  not  feel  that  current  training  requirements  development  met  the  needs  of  the 
commanders,  services,  or  training  commands.  JFCOM  develops  training  requirements 
through  a  working  group  that  involves  representatives  from  stakeholders,  but;  commander 
representatives  said  that  they  have  provided  little  input  into  the  development  of  joint  tasks. 

These  findings  demonstrate  that  in  order  for  training,  experimentation,  and 
assessment  to  work  successfully,  attitudes  and  culture  must  change  and  a  greater 
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understanding  of  the  differences  in  these  venues  be  developed.  However,  JFCOM  and 
service  leadership  have  yet  to  adequately  address  the  attitudes  and  cultural  barriers  affecting 
the  success  of  combined  venues. 

THE  FUTURE  OF  JOINT  TRAINING,  EXPERIMENTATION,  AND  ASSESSMENT 

One  of  the  more  significant  documents  concerning  mixed  venues  is  a  report  by  the 

Director,  Operational  Test  and  Evaluation  (DOT&E).  The  successful  future  of  DOT&E’s 

acquisition  responsibilities  rests  squarely  on  the  success  of  mixed  venues  and  JNTC. 

DOT&E’s  document.  Testing  in  a  Joint  Environment  Roadmap:  Strategic  Planning 

Guidance  Fiscal  Years  2006-2011—  Final  Report,  was  prepared  in  coordination  with  many 

mixed  venue  stakeholders.  It  was  published  in  November  2004  and  released  to  the  public  in 

March  2005.  The  scope  of  this  comprehensive  report  is  most  likely  a  reflection  of  the  level 

of  concern  DOT&E  has  with  the  results  of  recent  mixed  venues.  The  report  directs  DOT&E 

to  “provide  new  testing  capabilities”  and  identify  “the  changes  needed  to  ensure  that  test  and 

evaluation  is  conducted  in  a  joint  environment.”  Its  goal  is  to  enable  DOT&E  to  “conduct 

2 1 

adequate,  realistic,  and  timely  T&E  in  a  joint  environment” 

DOT&E’s  strategic  plan  does  provide  significant  recommendations  to  improve  many 
of  the  issues  plaguing  the  integration  of  experimentation  (testing)  into  training  events 
including:  developing  new  testing  capabilities,  methodologies,  and  documents  to  better 
acclimate  into  the  joint  environment;  increased  use  of  instrumentation  and  improved 
instrumentation  systems;  establishing  relationships  and  vetting  processes  of  stakeholders  to 
develop  better  venues;  and  pursuing  increased  funding  for  various  aspects  of  the  plan.  This 
additional  funding  will  undoubtedly  be  of  value  to  the  training  audience.  If  fully 
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implemented,  this  plan  will  establish  a  eapability  that  will  enable  mixed  venues  to  meet  the 
objeetives  of  the  acquisition  community. 

Many  of  the  proposed  reforms  will  positively  impact  joint  training  events.  However, 
the  military  should  stay  actively  involved  in  the  implementation  of  this  plan.  The  report 
primarily  focuses  on  ideas  that  will  strengthen  the  civilian  side  of  military  transformation  - 
the  development,  test  and  evaluation,  and  acquisition  of  joint  systems.  It  does  not  adequately 
represent  the  concerns  of  the  training  community  or  the  assessment  community  (nor  was  it 
designed  to). 

DOT&E  does  recognize  the  need  to  balance  requirements  and  maintain  the  focus  on 
joint  training  stating: 

Joint  forces,  once  assembled  for  testing,  training  and  experimentation,  should  be 
scheduled  with  balanced  and  coordinated  priorities  to  participate  in  support  of  testing, 
training,  and  experimentation  requirements  -  live  or  virtual. . . .  While  sharing  may  be 
a  strong  benefit,  it  cannot  dilute  the  primary  purpose  of  Joint  Training  which  remains 
to  prepare  the  operational  forces  to  fight  and  win  wars.^^ 

Further,  it  acknowledges  that  not  all  training  events  should  be  used  for  testing 

(experimentation).  DOT&E  recommends  that  if  training  exercises  can  not  meet  test 

objectives,  unique  test  events  will  need  to  be  conducted.  Support  for  this  philosophy 

demonstrates  a  willingness  to  keep  testing  (experimentation)  from  taking  priority  over 

training  objectives.  Finally,  the  report  recommends  changes  to  several  documents  including 

CJCS  Instruction  3500. OIB,  Joint  Training  Policy  for  the  Armed  Forces  of  the  United  States, 

dated  31  December  1999,  regarding  the  clarification  of  the  use  of  current  and  developmental 

doctrine  and  TTP. 

However,  this  is  as  far  as  the  report  goes  in  promoting  the  training  aspects  of  mixed 
events.  While  it  provides  changes  to  several  current  documents,  most  of  the  changes  direct 
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the  incorporation  of  testing  efforts  into  joint  training  environments.  It  does  not  address  many 
of  the  issues  discussed  in  this  paper  such  as  the  resolution  of  leadership,  priorities,  execution, 
and  task  development  conflicts  among  combatant  commanders,  component  commanders. 
Chairman  Joint  Chiefs  of  Staff,  and  JFCOM.  It  does  not  mention  how  to  resolve  issues  of 
repetition,  feedback,  and  combining  service  and  joint  training.  Finally,  it  does  not  address 
cultural  issues  such  as  failure  acceptance,  over-reliance  on  simplified  quick-look  reports,  or 
time  for  proper  exercise  management. 

Included  in  DOT&E’s  report  are  comments  of  the  plan  from  various  stakeholders 
including  the  service  secretaries  and  JFCOM.  Each  one  concurred  with  the  plan  and  not  one 
military  stakeholder  expressed  concern  over  experimentation  (testing)  or  assessment 
requirements  overshadowing  training  requirements. 


RECOMMENDATIONS 

Books  such  as  The  Role  of  Experimentation  in  Building  Future  Naval  Forces  and 
Code  of  Best  Practices  for  Experimentation,  exercise  post-event  reports,  and  feedback  from 
participants  (trainers,  experimenters,  and  assessors)  document  the  shortfalls  in  the  current 
mixed  venue  philosophy.  The  DoD  acquisition  community,  as  represented  by  the  office  of 
the  DOT&E,  has  heeded  this  feedback  and  has  developed  a  comprehensive  roadmap  that 
addresses  many  significant  test/experimentation-related  issues. 

Although  supported  by  DoD  senior  leadership,  not  every  event  has  to  be  or  should  be 
a  combined  event.  Planners  should  consider  various  types  of  events  and  establish  a  priority 
for  each  -  be  it  training,  experimentation,  or  assessment.  Combatant  or  component 
commanders  should  lead  training  events  and  JECOM  should  lead  experimentation  and 
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assessment  events.  Additionally,  the  serviees  must  be  allowed  to  maintain  their  own  service- 
speeifie  events  to  prepare  their  forees  for  “graduate-level”  joint  events.  Experiments  ean  be 
performed  during  training  events  and  viee-versa,  but  the  emphasis  must  be  on  one  or  the 
other.  Venues  must  elarify  whether  doetrine  and  TTP  adherenee  or  development  is  the 
priority  and  assessment  efforts  must  eompliment,  not  eontradict  the  intended  effort.  Training 
events  must  provide  for  a  more  eomprehensive  feedbaek  experienee  for  the  eommanders,  the 
exereise  JFC,  his  staff,  and  partieipating  forees.  The  GAO  highlighted  two  signifieant 
eultural  challenges  that  should  be  addressed: 

(1)  [Establish]  effective  partnerships  with  program  stakeholders,  such  as  the 
services  and  combatant  commands,  through  comprehensive  communication  and 
coordination;  and  (2)  [develop]  joint  training  requirements  that  meet  combatant 
commands’  mission  needs.  Both  these  challenges,  if  left  unaddressed,  have  the 
potential  for  eroding  support  among  program  stakeholders,  which  in  turn  places  the 

23 

goals  of  the  Training  Transformation  Program  at  risk. 

In  its  response  to  the  GAO,  the  DoD  stated: 

[I]n  FY06,  many  JNTC  events  will  transition  from  centralized  JFCOM  planning 
and  execution  to  decentralized  planning  and  execution  by  Service-  and  CC 
[combatant  commander] -nominated  training  programs  assisted  by  JFCOM. 

Additionally,  JFCOM  should  make  more  comprehensive  use  of  the  Joint  Training 
System  to  better  reflect  CC  [combatant  commander]  training  requirements  and  buy- 

in.^4 

While  DoD’s  response  sounds  encouraging,  it  will  be  up  to  military  leadership  to 
break  the  cultural  barriers  that  influence  the  effectiveness  of  combined  venues. 

DOT&E’s  Strategic  Planning  Guidance  is  an  attempt  to  remedy  shortfalls  in  testing 
guidance.  The  intent  of  this  paper  is  not  to  criticize  these  efforts.  On  the  contrary,  DOT&E 
should  be  applauded  for  addressing  the  many  problems  of  current  practices.  However,  if  left 
unchecked,  the  acquisition  community  will  dictate  the  design  of  future  mixed  venues  further 
weakening  the  commanders’  control  over  training.  Combatant  and  component  commanders 
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must  work  more  closely  with  JFCOM  and  other  assessment  organizations  to  develop  the 
right  mix  of  joint  training,  experimentation,  and  assessment  venues.  Military  leadership  must 
develop  a  similar  roadmap  that  protects  the  sanctity  of  service  and  joint  training  objectives. 


CONCLUSION 

This  paper  attempted  to  show  that  current  doctrine  and  guidance  are  inadequate  to 
fully  realize  the  potential  efficiencies  expected  by  combining  joint  training,  experimentation, 
and  assessment,  especially  in  service-specific  events. 

The  shortfalls  of  combined  joint  training,  experimentation,  and  assessment  venues 
have  been  well  documented.  Several  studies  provide  recommendations  regarding  the 
integration  of  experimentation  and  assessment.  In  any  large,  bureaucratic  organization, 
change  is  difficult  to  implement.  The  U.S.  military  is  currently  challenged  with  addressing 
transformational  changes  not  only  in  organization,  but  in  policy,  doctrine,  funding,  manning, 
mission,  procedures,  and  culture  as  well.  The  effects  of  the  implementation  of  these  changes 
come  together,  in  many  cases  for  the  first  time,  in  joint  training,  experimentation,  and 
assessment  events.  Until  the  implementation  of  these  changes  matures  and  transformation 
becomes  culturally  accepted,  all  mixed  venue  stakeholders  will  be  affected.  The  component 
commander  exercises  mentioned  previously  fulfill  a  significant  training  requirement  of  the 
combatant  commander.  During  these  events,  his  training  requirements  must  always  be  at  the 
forefront  with  all  other  efforts  subordinate  to  those  requirements.  If  additional  events  need  to 
be  developed  to  satisfy  experimentation  and  assessment  requirements,  then  DoD  must  be 
willing  to  invest  in  them. 
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As  would  be  expected,  the  first  attempts  at  combining  these  efforts  will  incur 
problems.  Many  individuals  involved  with  the  planning  and  implementation  of  training 
venues  believe  that  combining  joint  training  with  experimentation  and  assessment  will  not 
work.  They  are  both  right  and  wrong.  Using  current  practices,  guidance,  and  policy  as  a 
measure,  they  are  right.  Too  many  impediments  are  in  place  to  prevent  mixed  venues  from 
satisfying  all  stakeholders’  needs  including  conflicting  objectives  and  priorities,  inadequate 
policies,  and  organizational  culture.  The  benefits  of  combined  venues  are  obvious;  however, 
so  too,  are  the  problems.  What  is  not  obvious,  but  achievable,  is  how  to  realize  the 
efficiencies  while  maximizing  the  effectiveness  of  all  training  events.  If  senior  leadership 
will  commit  to  implementing  the  recommendations  provided  in  The  Role  of  Experimentation 
in  Building  Future  Naval  Forces,  Code  of  Best  Practice  for  Experimentation,  GAO  report 
05-548,  and  DoD’s  Strategic  Planning  Guidance,  mixed  venues  may  one  day  be  capable  of 
satisfying  the  objectives  of  all  stakeholders. 

In  a  memorandum  to  Secretary  Rumsfeld  dated  December  13,  2002,  Thomas  P. 
Christie,  DOT&E,  stated: 

While  other  recommendations  are  forthcoming,  I  believe  one  item  should  top  the 
list: 

To  strengthen  our  joint  warfighting  capabilities,  the  Department  should  not  only 
“train  as  we  fight”  but  also  “test  as  we  fight.”^^ 

While  DoD  should  and  can  “train  like  we  fight”  and  “test  like  we  fight,”  this  paper 
has  attempted  to  show  that  DoD  can  not  yet  “train  and  test  like  we  fight.” 

The  overwhelming  superiority  and  success  of  U.S.  military  forces  in  conventional 
warfare  and  lack  of  a  peer  competitor  have  created  a  false  sense  of  security  in  regard  to  the 
effectiveness  of  current  training  venues  (both  service  and  joint).  Mounting  training 
requirements  and  ever-increasing  technology  of  systems  continue  to  strain  available  training 
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opportunities.  Future  training  requirements  will  inelude  an  increased  emphasis  on 
asymmetric  warfare  and  interagency  coordination.  Designing  an  appropriate  spectrum  of 
training,  experimentation,  and  assessment  events  will  require  inputs  from  all  stakeholders. 
DOT&E  has  developed  a  comprehensive  plan  to  adapt  and  integrate  into  joint  training 
events.  The  military  must  produce  a  similar  roadmap  that  addresses  combatant  and 
component  commander  responsibilities.  Proper  integration  of  objectives,  recognition  of 
contradicting  requirements,  and  a  willingness  by  leadership  to  break  cultural  boundaries  will 
alleviate  many  current  compromises  and  conflicts,  and  promote  a  more  harmonious 
integration  of  experimentation  and  assessment  efficiency  with  training  effectiveness. 
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